# Weight-Sharing Architecture

ALBERT is a lightweight pre-trained language model based on the Transformer architecture, trained on English text through self-supervised learning with parameter-sharing features to reduce memory usage.

ALBERT is a lightweight BERT variant pre-trained on English corpora, reducing memory usage through parameter sharing, supporting masked language modeling and sentence order prediction tasks.